Adjustment device and method of operating the same

ABSTRACT

An adjustment device includes a wearable electronic device and a case in which the wearable electronic device is disposed (e.g., seated). The wearable electronic device includes displays (e.g., display apparatuses) which display virtual images for a left eye and a right eye of a user, screen display portions which transmit light sources generated by the displays to the left eye and the right eye, and eye tracking cameras for the left eye and the right eye. The case includes a stator which fixes the wearable electronic device, and a focal lens which is disposed within an eye relief of the fixed wearable electronic device and forms each of images of the virtual images output from the screen display portions of the wearable electronic device on a portion of the case.

This application is a national stage application of InternationalApplication No. PCT/KR2022/007280 designating the United States, filedon May 27, 2022, which claims priority to Korean Patent Application No.10-2021-0100480, filed on Jul. 30, 2021, and all the benefits accruingtherefrom under 35 U.S.C. § 119, the content of which in its entirety isherein incorporated by reference.

BACKGROUND 1. Field

Embodiments of the invention relate to an adjustment device and a methodof operating the adjustment device.

2. Description of Related Art

A wearable electronic device for providing an augmented reality (“AR”)service is being introduced on the market. The AR service is a serviceof superimposing a virtual image having supplementary information on areal-world image seen by a user and showing a superimposition result,and may provide a user with a virtual object image including contentrelated to a real object identified from the real-world image. Thewearable electronic device for providing the AR service may beconfigured in a form of a head-mounted display (“HMD”), for example.

SUMMARY

An adjustment device in an embodiment includes a wearable electronicdevice, and a case in which the wearable electronic device is disposed(e.g., seated). The wearable electronic device may include displays(e.g., display apparatuses) which correspond to a left eye and a righteye of a user and display virtual images, screen display portions whichcorrespond to the left eye and the right eye and transmit light sourcesgenerated by the display apparatuses to the left eye and the right eye,and eye tracking cameras which correspond to the left eye and the righteye. The case may include a stator which fixes the wearable electronicdevice, and a focal lens which is disposed within an eye relief of thefixed wearable electronic device and forms each of images of the virtualimages output from the screen display portions of the wearableelectronic device on a portion of the case.

A method of operating an adjustment device including a case, in which awearable electronic device including screen display portions whichcorrespond to a left eye and a right eye of a user and eye trackingcameras which correspond to the left eye and the right eye is disposed,includes transmitting virtual images for measuring a deviation betweenthe screen display portions to the screen display portions, projectingthe virtual images onto a portion of the case through focus lensesdisposed in the case, capturing the projected virtual images by the eyetracking cameras, and adjusting the screen display portions based on acomparison result of the captured images.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of embodiments of theinvention will be more apparent from the following detailed description,taken in conjunction with the accompanying drawings, in which:

FIG. 1 is a block diagram of an embodiment of an electronic device in anetwork environment;

FIG. 2 is a diagram illustrating an embodiment of a structure of awearable electronic device;

FIG. 3 is a diagram illustrating an embodiment of an operation of an eyetracking camera included in a wearable electronic device;

FIG. 4 is a diagram illustrating an embodiment of operations of adisplay and screen display portions of a wearable electronic device;

FIG. 5 is a diagram illustrating an embodiment of a structure of anadjustment device including a wearable electronic device and a case;

FIG. 6A is a diagram illustrating an embodiment of a virtual imageprojected by an adjustment device, and 6B is an enlarged view of aportion indicated by a dot-dash line in FIG. 6A;

FIG. 7 is a diagram illustrating another embodiment of a structure of anadjustment device including a wearable electronic device and a case;

FIG. 8A is a diagram illustrating an embodiment of a method of adjustingscreen display portions of a wearable electronic device using a drivingdevice included in a case, and FIG. 8B is an enlarged view of a portionindicated by a dotted line in FIG. 8A;

FIGS. 9A to 9C and 10 are diagrams illustrating an embodiment of amethod of adjusting screen display portions of a wearable electronicdevice using a driving device included in the wearable electronicdevice;

FIG. 11 is a flowchart illustrating an embodiment of a method ofoperating an adjustment device;

FIG. 12 is a flowchart illustrating another embodiment of a method ofoperating an adjustment device; and

FIG. 13 is a flowchart illustrating an embodiment of a method ofadjusting screen display portions using multifocal plane.

DETAILED DESCRIPTION

Hereinafter, various embodiments will be described in detail withreference to the accompanying drawings. When describing the embodimentswith reference to the accompanying drawings, like reference numeralsrefer to like elements and a repeated description related thereto willbe omitted.

The invention now will be described more fully hereinafter withreference to the accompanying drawings, in which various embodiments areshown. This invention may, however, be embodied in many different forms,and should not be construed as limited to the embodiments set forthherein. Rather, these embodiments are provided so that this inventionwill be thorough and complete, and will fully convey the scope of theinvention to those skilled in the art. Like reference numerals refer tolike elements throughout.

It will be understood that when an element is referred to as being “on”another element, it can be directly on the other element or interveningelements may be therebetween. In contrast, when an element is referredto as being “directly on” another element, there are no interveningelements present.

It will be understood that, although the terms “first,” “second,”“third” etc. may be used herein to describe various elements,components, regions, layers and/or sections, these elements, components,regions, layers and/or sections should not be limited by these terms.These terms are only used to distinguish one element, component, region,layer or section from another element, component, region, layer orsection. Thus, “a first element,” “component,” “region,” “layer” or“section” discussed below could be termed a second element, component,region, layer or section without departing from the teachings herein.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used herein, thesingular forms “a,” “an,” and “the” are intended to include the pluralforms, including “at least one,” unless the content clearly indicatesotherwise. “Or” means “and/or.” As used herein, the term “and/or”includes any and all combinations of one or more of the associatedlisted items. It will be further understood that the terms “comprises”and/or “comprising,” or “includes” and/or “including” when used in thisspecification, specify the presence of stated features, regions,integers, steps, operations, elements, and/or components, but do notpreclude the presence or addition of one or more other features,regions, integers, steps, operations, elements, components, and/orgroups thereof.

Furthermore, relative terms, such as “lower” or “bottom” and “upper” or“top,” may be used herein to describe one element's relationship toanother element as illustrated in the Figures. It will be understoodthat relative terms are intended to encompass different orientations ofthe device in addition to the orientation depicted in the Figures. In anembodiment, when the device in one of the figures is turned over,elements described as being on the “lower” side of other elements wouldthen be oriented on “upper” sides of the other elements. The term“lower,” can therefore, encompasses both an orientation of “lower” and“upper,” depending on the particular orientation of the figure.Similarly, when the device in one of the figures is turned over,elements described as “below” or “beneath” other elements would then beoriented “above” the other elements. The terms “below” or “beneath” can,therefore, encompass both an orientation of above and below.

“About” or “approximately” as used herein is inclusive of the statedvalue and means within an acceptable range of deviation for theparticular value as determined by one of ordinary skill in the art,considering the measurement in question and the error associated withmeasurement of the particular quantity (i.e., the limitations of themeasurement system). The term “about” can mean within one or morestandard deviations, or within ±30%, 20%, 10%, 5% of the stated value,for example.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which this invention belongs. It will befurther understood that terms, such as those defined in commonly useddictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art and theinvention, and will not be interpreted in an idealized or overly formalsense unless expressly so defined herein. In the description, terms suchas “module” may mean “circuitry block”. Further, a term “display” maymean a physical structure (e.g., display apparatus) which displays animage.

FIG. 1 is a block diagram illustrating an electronic device 101 in anetwork environment 100 according to various embodiments. Referring toFIG. 1 , the electronic device 101 in the network environment 100 maycommunicate with an electronic device 102 via a first network 198 (e.g.,a short-range wireless communication network), or communicate with atleast one of an electronic device 104 or a server 108 via a secondnetwork 199 (e.g., a long-range wireless communication network). In anembodiment, the electronic device 101 may communicate with theelectronic device 104 via the server 108. In an embodiment, theelectronic device 101 may include a processor 120, a memory 130, aninput module 150, a sound output module 155, a display module 160, anaudio module 170, and a sensor module 176, an interface 177, aconnecting terminal 178, a haptic module 179, a camera module 180, apower management module 188, a battery 189, a communication module 190,a subscriber identification module (“SIM”) 196, or an antenna module197. In some alternative embodiments, at least one of the components(e.g., the connecting terminal 178) may be omitted from the electronicdevice 101, or one or more other components may be added in theelectronic device 101. In some embodiments, some of the components(e.g., the sensor module 176, the camera module 180, or the antennamodule 197) may be unitarily integrated as a single component (e.g., thedisplay module 160).

In an embodiment, the processor 120 may execute software (e.g., aprogram 140) to control at least one other component (e.g., a hardwareor software component) of the electronic device 101 connected to theprocessor 120, and may perform various data processing or computation.In an embodiment, as at least a part of data processing or computation,the processor 120 may store a command or data received from anothercomponent (e.g., the sensor module 176 or the communication module 190)in a volatile memory 132, process the command or the data stored in thevolatile memory 132, and store resulting data in a non-volatile memory134. In an embodiment, the processor 120 may include a main processor121 (e.g., a central processing unit (“CPU”) or an application processor(“AP”)), or an auxiliary processor 123 (e.g., a graphics processing unit(“GPU”), a neural processing unit (“NPU”), an image signal processor(“ISP”), a sensor hub processor, or a communication processor (“CP”))that is operable independently from, or in conjunction with the mainprocessor 121. In an embodiment, when the electronic device 101 includesthe main processor 121 and the auxiliary processor 123, the auxiliaryprocessor 123 may be adapted to consume less power than the mainprocessor 121 or to be specific to a specified function. The auxiliaryprocessor 123 may be implemented separately from the main processor 121or as a part of the main processor 121.

The auxiliary processor 123 may control at least some of functions orstates related to at least one (e.g., the display module 160, the sensormodule 176, or the communication module 190) of the components of theelectronic device 101, instead of the main processor 121 while the mainprocessor 121 is in an inactive (e.g., sleep) state or along with themain processor 121 while the main processor 121 is an active state(e.g., executing an application). In an embodiment, the auxiliaryprocessor 123 (e.g., an ISP or a CP) may be implemented as a portion ofanother component (e.g., the camera module 180 or the communicationmodule 190) that is functionally related to the auxiliary processor 123.In an embodiment, the auxiliary processor 123 (e.g., an NPU) may includea hardware structure specified for artificial intelligence (“AI”) modelprocessing. In an embodiment, an AI model may be generated by machinelearning. Such learning may be performed by the electronic device 101 inwhich AI is performed, or performed via a separate server (e.g., theserver 108), for example. In an embodiment, learning algorithms mayinclude, but are not limited to supervised learning, unsupervisedlearning, semi-supervised learning, or reinforcement learning, forexample. The AI model may include a plurality of artificial neuralnetwork layers. In an embodiment, an artificial neural network mayinclude a deep neural network (“DNN”), a convolutional neural network(“CNN”), a recurrent neural network (“RNN”), a restricted Boltzmannmachine (“RBM”), a deep belief network (“DBN”), and a bidirectionalrecurrent deep neural network (“BRDNN”), a deep Q-network, or acombination of two or more thereof, for example, but is not limitedthereto. The AI model may additionally or alternatively include asoftware structure other than the hardware structure.

In an embodiment, the memory 130 may store various data used by at leastone component (e.g., the processor 120 or the sensor module 176) of theelectronic device 101. In an embodiment, the various data may includesoftware (e.g., the program 140) and input data or output data for acommand related thereto, for example. The memory 130 may include thevolatile memory 132 or the non-volatile memory 134. In an embodiment,the non-volatile memory 134 may include an internal memory 136 and anexternal memory 138.

In an embodiment, the program 140 may be stored as software in thememory 130, and may include an operating system (“OS”) 142, middleware144, or an application 146, for example.

The input module 150 may receive a command or data to be used by anothercomponent (e.g., the processor 120) of the electronic device 101, fromthe outside (e.g., a user) of the electronic device 101. In anembodiment, the input module 150 may include a microphone, a mouse, akeyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen),for example.

The sound output module 155 may output a sound signal to the outside ofthe electronic device 101. In an embodiment, the sound output module 155may include a speaker or a receiver. The speaker may be used for generalpurposes, such as playing multimedia or playing record. The receiver maybe used to receive an incoming call. In an embodiment, the receiver maybe implemented separately from the speaker or as a part of the speaker.

The display module 160 may visually provide information to the outside(e.g., a user) of the electronic device 101. In an embodiment, thedisplay module 160 may include a control circuit for controlling adisplay, a hologram device, or a projector and control circuitry tocontrol a corresponding one of the display, the hologram device, and theprojector. In an embodiment, the display module 160 may include a touchsensor adapted to detect a touch, or a pressure sensor adapted tomeasure the intensity of force incurred by the touch.

The audio module 170 may convert a sound into an electric signal or viceversa. In an embodiment, the audio module 170 may obtain the sound viathe input module 150 or output the sound via the sound output module 155or an external electronic device (e.g., the electronic device 102 suchas a speaker or a headphone) directly or wirelessly connected to theelectronic device 101.

The sensor module 176 may detect an operational state (e.g., power ortemperature) of the electronic device 101 or an environmental state(e.g., a state of a user) external to the electronic device 101, andgenerate an electrical signal or data value corresponding to thedetected state. In an embodiment, the sensor module 176 may include agesture sensor, a gyro sensor, an atmospheric pressure sensor, amagnetic sensor, an acceleration sensor, a grip sensor, a proximitysensor, a color sensor, an infrared (“IR”) sensor, a biometric sensor, atemperature sensor, a humidity sensor, or a fingerprint sensor, forexample.

The interface 177 may support one or more specified protocols to be usedfor the electronic device 101 to be coupled with the external electronicdevice (e.g., the electronic device 102) directly (e.g., by a wireconnection) or wirelessly (e.g., without a wire connection). In anembodiment, the interface 177 may include a high-definition multimediainterface (“HDMI”), a universal serial bus (“USB”) interface, a securedigital (“SD”) card interface, or an audio interface, for example.

The connecting terminal 178 may include a connector via which theelectronic device 101 may be physically connected to an externalelectronic device (e.g., the electronic device 102). In an embodiment,the connecting terminal 178 may include an HDMI connector, a USBconnector, an SD card connector, or an audio connector (e.g., aheadphone connector), for example.

The haptic module 179 may convert an electric signal into a mechanicalstimulus (e.g., a vibration or a movement) or an electrical stimuluswhich may be recognized by a user via tactile sensation or kinestheticsensation of the user. In an embodiment, the haptic module 179 mayinclude a motor, a piezoelectric element, or an electric stimulator, forexample.

The camera module 180 may capture a still image and moving images. In anembodiment, the camera module 180 may include one or more lenses, imagesensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to theelectronic device 101. In an embodiment, the power management module 188may be implemented as at least a part of a power management integratedcircuit (“PMIC”), for example.

The battery 189 may supply power to at least one component of theelectronic device 101. In an embodiment, the battery 189 may include aprimary cell which is not rechargeable, a secondary cell which isrechargeable, or a fuel cell, for example.

The communication module 190 may support establishing a direct (e.g.,wired) communication channel or a wireless communication channel betweenthe electronic device 101 and the external electronic device (e.g., theelectronic device 102, the electronic device 104, or the server 108) andperforming communication via the established communication channel. Thecommunication module 190 may include one or more communicationprocessors that are operable independently of the processor 120 (e.g.,an AP) and that support a direct (e.g., wired) communication or awireless communication. In an embodiment, the communication module 190may include a wireless communication module 192 (e.g., a cellularcommunication module, a short-range wireless communication module, or aglobal navigation satellite system (“GNSS”) communication module) or awired communication module 194 (e.g., a local area network (“LAN”)communication module, or a power line communication (“PLC”) module). Acorresponding one of these communication modules may communicate withthe external electronic device 104 via the first network 198 (e.g., ashort-range communication network, such as Bluetooth™, wireless-fidelity(“Wi-Fi”) direct, or infrared data association (“IrDA”)) or the secondnetwork 199 (e.g., a long-range communication network, such as a legacycellular network, a 5G network, a next-generation communication network,the Internet, or a computer network (e.g., a LAN or a wide area network(“WAN”))). These various types of communication modules may beimplemented as a single component (e.g., a single chip), or may beimplemented as multi components (e.g., multi chips) separated from eachother. The wireless communication module 192 may identify andauthenticate the electronic device 101 in a communication network, suchas the first network 198 or the second network 199, using subscriberinformation (e.g., international mobile subscriber identity (“IMSI”))stored in the SIM 196.

The wireless communication module 192 may support a 5G networksubsequent to a 4G network, and next-generation communicationtechnology, e.g., new radio (“NR”) access technology. The NR accesstechnology may support enhanced mobile broadband (“eMBB”), massivemachine type communications (“mMTC”), or ultra-reliable and low-latencycommunications (“URLLC”). The wireless communication module 192 maysupport a high-frequency band (e.g., a millimeter wave (“mmWave”) band)to achieve a high data transmission rate, for example. The wirelesscommunication module 192 may support various technologies for securingperformance on a high-frequency band, such as beamforming, massivemultiple-input and multiple-output (“MIMO”), full dimensional MIMO(“FD-MIMO”), an array antenna, analog beam-forming, or a large scaleantenna. The wireless communication module 192 may support variousrequirements specified in the electronic device 101, an externalelectronic device (e.g., the electronic device 104), or a network system(e.g., the second network 199). In an embodiment, the wirelesscommunication module 192 may support a peak data rate (e.g., about 20gigabits per second (Gbps) or more) for implementing eMBB, loss coverage(e.g., about 164 decibel (dB) or less) for implementing mMTC, or U-planelatency (e.g., about 0.5 millisecond (ms) or less for each of downlink(“DL”) and uplink (“UL”), or a round trip of about 1 ms or less) forimplementing URLLC.

The antenna module 197 may transmit or receive a signal or power to orfrom the outside (e.g., the external electronic device) of theelectronic device 101. In an embodiment, the antenna module 197 mayinclude an antenna including a radiating element including a conductivematerial or a conductive pattern formed in or on a substrate (e.g., aprinted circuit board (“PCB”)). In an embodiment, the antenna module 197may include a plurality of antennas (e.g., array antennas). In such acase, at least one antenna appropriate for a communication scheme usedin a communication network, such as the first network 198 or the secondnetwork 199, may be selected by the communication module 190 from theplurality of antennas, for example. The signal or the power may betransmitted or received between the communication module 190 and theexternal electronic device via the at least one selected antenna. In anembodiment, another component (e.g., a radio frequency integratedcircuit (“RFIC”)) other than the radiating element may be additionallyformed as a part of the antenna module 197.

According to various embodiments, the antenna module 197 may form ammWave antenna module. In an embodiment, the mmWave antenna module mayinclude a PCB, an RFIC disposed on a first surface (e.g., a bottomsurface) of the PCB or adjacent to the first surface and capable ofsupporting a designated a high-frequency band (e.g., the mmWave band),and a plurality of antennas (e.g., array antennas) disposed on a secondsurface (e.g., a top or a side surface) of the PCB, or adjacent to thesecond surface and capable of transmitting or receiving signals in thedesignated high-frequency band.

At least some of the above-described components may be coupled mutuallyand communicate signals (e.g., commands or data) therebetween via aninter-peripheral communication scheme (e.g., a bus, general purposeinput and output (“GPIO”), serial peripheral interface (“SPI”), ormobile industry processor interface (“MIPI”)).

In an embodiment, commands or data may be transmitted or receivedbetween the electronic device 101 and the external electronic device 104via the server 108 coupled with the second network 199. Each of theexternal electronic devices 102 or 104 may be a device of the same typeas or a different type from the electronic device 101. The externalelectronic device 102 may be a wearable electronic device 200, forexample.

In an embodiment, all or some of operations to be executed by theelectronic device 101 may be executed at one or more external electronicdevices (e.g., the external devices 102 and 104, and the server 108). Inan embodiment, when the electronic device 101 needs to perform afunction or a service automatically, or in response to a request from auser or another device, for example, the electronic device 101, insteadof, or in addition to, executing the function or the service, mayrequest the one or more external electronic devices (e.g., the externaldevices 102 and 104, and the server 108) to perform at least a part ofthe function or the service. The one or more external electronic devices(e.g., the external devices 102 and 104, and the server 108) receivingthe request may perform the at least a part of the function or theservice requested, or an additional function or an additional servicerelated to the request, and may transfer an outcome of the performing tothe electronic device 101. The electronic device 101 may provide theoutcome, with or without further processing of the outcome, as at leasta part of a reply to the request. To that end, a cloud computing,distributed computing, mobile edge computing (“MEC”), or client-servercomputing technology may be used, for example. The electronic device 101may provide ultra-low-latency services using distributed computing ormobile edge computing, for example. In an embodiment, the externalelectronic device 104 may include an Internet-of-things (“IoT”) device.The server 108 may be an intelligent server using machine learningand/or a neural network. In an embodiment, the external electronicdevice 104 or the server 108 may be included in the second network 199.The electronic device 101 may be applied to intelligent services (e.g.,smart home, smart city, smart car, or healthcare) based on 5Gcommunication technology or IoT-related technology.

FIG. 2 is a diagram illustrating an embodiment of a structure of awearable electronic device.

Referring to FIG. 2 , the wearable electronic device 200 (e.g., theelectronic device 101 or 102 of FIG. 1 ) may be disposed (e.g., worn) ona face of a user to provide the user with an image associated with anaugmented reality (“AR”) service and/or a virtual reality (“VR”)service.

In an embodiment, the wearable electronic device 200 may include a firstdisplay 205, a second display 210, screen display portions 215 a and 215b, input optical members 220 a and 220 b, a first transparent member 225a, a second transparent member 225 b, lighting units 230 a and 230 b, afirst PCB 235 a, a second PCB 235 b, a first hinge 240 a, a second hinge240 b, an imaging camera 245, a plurality of microphones (e.g., a firstmicrophone 250 a, a second microphone 250 b, and a third microphone 250c), a plurality of speakers (e.g., a first speaker 255 a, and a secondspeaker 255 b), a battery 260, a first recognition camera 265 a, asecond recognition camera 265 b, a first eye tracking camera 270 a, anda second eye tracking camera 270 b.

In an embodiment, a display (e.g., the first display 205 and the seconddisplay 210) may include a liquid crystal display (“LCD”), a digitalmirror device (“DMD”), or a liquid crystal on silicon (“LCoS”), anorganic light-emitting diode (“OLED”), a micro light-emitting diode(“micro LED”), or the like, for example. Although not shown in thedrawings, when the first display 205 and/or the second display 210 isone of an LCD, a DMD, and an LCoS, the wearable electronic device 200may include a light source which emits light to a screen output area ofthe first display 205 and/or the second display 210. In an embodiment,when the first display 205 and/or the second display 210 is capable ofgenerating light by itself (when the first display 205 and/or the seconddisplay 210 is either an OLED or a micro-LED, for example), the wearableelectronic device 200 may provide a virtual image with a relatively highquality to the user even though a separate light source is not included.In an embodiment, when the first display 205 and/or the second display210 is implemented as an OLED or a micro-LED, a light source may beunnecessary, and accordingly the wearable electronic device 200 may belightened, for example. Hereinafter, the first display 205 and/or thesecond display 210 capable of generating light by itself may be alsoreferred to as a “self-luminous display”, and description will be madeon the assumption of the self-luminous display.

The first display 205 and/or the second display 210 according to variousembodiments may include at least one micro-LED. In an embodiment, themicro-LED may express red (R), green (G), and blue (B) by emitting lightby itself, and a single chip may implement a single pixel (e.g., one ofR, G, and B pixels) because the micro-LED is relatively small in size(e.g., about 100 micrometer (μm) or less), for example. Accordingly, itmay be possible to provide a high resolution without a backlight unit(“BLU”), when the first display 205 and/or the second display 210 isimplemented as a micro-LED. However, the invention is not limitedthereto, and a single chip may be implemented by a plurality of pixelsincluding R, G, and B pixels. However, the invention is not limitedthereto, and the plurality of pixels may include various other colorpixels. The first display 205 and/or the second display 210 may be alsoreferred to as a “light source”.

A structure and an operation of the first display 205 and/or the seconddisplay 210 will be described in more detail below with reference toFIG. 4 .

In an embodiment, the first display 205 and/or the second display 210may include pixels for displaying a virtual image. The first display 205and/or the second display 210 may further include infrared pixels thatemit infrared light.

In an embodiment, the first display 205 and/or the second display 210may further include light-receiving pixels (e.g., photo sensor pixels)that are disposed between pixels, receive light reflected from eyes of auser, convert the received light to electrical energy, and output theelectrical energy. A light-receiving pixel may be also referred to as an“eye tracking sensor”. The eye tracking sensor (e.g., an eye trackingsensor 315 of FIG. 3 ) may sense infrared light generated by reflectinglight emitted by an infrared pixel included in the first display 205and/or the second display 210 by eyes of a user.

The wearable electronic device 200 may detect a gaze direction (e.g., amovement of a pupil) of the user, using light-receiving pixels 315. Inan embodiment, the wearable electronic device 200 may detect and track agaze direction of each of a right eye and a left eye of the user throughone or more light-receiving pixels 315 of the first display 205 and oneor more light-receiving pixels 315 of the second display 210, forexample. The wearable electronic device 200 may also determine a centralposition of a virtual image 610 (refer to FIGS. 6A and 6B) according tothe gaze directions (e.g., directions in which pupils of the right eyeand the left eye of the user gaze) detected through the one or morelight-receiving pixels 315.

The wearable electronic device 200 may include the first display 205and/or the second display 210, the first transparent member 225 a and/orthe second transparent member 225 b. A user may use the wearableelectronic device 200 while wearing the wearable electronic device 200on a face of the user. In an embodiment, the first transparent member225 a may face the right eye of the user, and the second transparentmember 225 b may face the left eye of the user. According to variousembodiments, when the first display 205 and/or the second display 210are transparent, the first display 205 and/or the second display 210 mayface the eyes of the user to configure the screen display portions 215 aand 215 b.

The first display 205 and the second display 210 may each include afirst control circuit (not shown). The first control circuit may controlthe first display 205 and the second display 210. The first controlcircuit may control an operation of a liquid crystal element of atransparent cover (not shown) included in each of the first display 205and the second display 210. In an embodiment, light emitted from thefirst display 205 and/or the second display 210 may reach the screendisplay portion 215 a formed on the first transparent member 225 a thatfaces the right eye of the user, and the screen display portion 215 bformed on the second transparent member 225 b that faces the left eye ofthe user, by passing through a lens (not shown) and a waveguide (e.g., adisplay waveguide 350 and an eye tracking waveguide 360 of FIG. 3 ).

The lens (not shown) may be disposed in front of the first display 205and/or the second display 210. The lens (not shown) may include aconcave lens and/or a convex lens. In an embodiment, the lens (notshown) may include a projection lens (e.g., a projection lens 325 ofFIG. 3 ), or a collimation lens (not shown), for example.

In an embodiment, the light emitted from the first display 205 and/orthe second display 210 may be guided by the display waveguide 350 and/orthe eye tracking waveguide 360 through the input optical members 220 aand 220 b. Light moving into the display waveguide 350 and/or the eyetracking waveguide 360 may be guided toward eyes of a user through anoutput optical member (e.g., an output optical member 340 of FIG. 3 ).The screen display portions 215 a and 215 b may be determined based onlight emitted toward an eye of a user (e.g., an eye 301 of the user ofFIG. 3 ).

In an embodiment, the light emitted from the first display 205 and/orthe second display 210 may be reflected from a grating area of thedisplay waveguide 350 and/or the eye tracking waveguide 360 formed inthe input optical member 220 a, 220 b and the screen display portion 215a, 215 b, and may be transmitted to the eye 301 of the user, forexample.

In an embodiment, the screen display portion 215 a, 215 b or atransparent member (e.g., the first transparent member 225 a and thesecond transparent member 225 b) may include a reflective lens, and alens including the display waveguide 350 and/or the eye trackingwaveguide 360. The display waveguide 350 and the eye tracking waveguide360 may function to transmit a light source generated by the firstdisplay 205 and/or the second display 210 to eyes of the user, and maybe also referred to as an “optical waveguide”. Hereinafter, an “opticalwaveguide” or “waveguide” may correspond to the screen display portions215 a and 215 b. The screen display portions 215 a and 215 b may be apath through which external light is incident, totally reflected, andemitted, and may be distinguished from the first transparent member 225a and the second transparent member 225 b through which external lightis simply reflected or transmitted.

In an embodiment, the screen display portions 215 a and 215 b mayinclude glass, plastic, or a polymer, and may have a nanopattern formedon one surface of the inside or outside, that is, a grating structure ofa polygonal or curved shape. In an embodiment, light incident to one endof the screen display portions 215 a and 215 b through the input opticalmember 220 a, 220 b may be propagated inside the display waveguide 350by the nanopattern to be provided to the user. In an embodiment, thescreen display portions 215 a and 215 b including a freeform prism mayprovide incident light to a user through a reflection mirror, forexample.

The screen display portions 215 a and 215 b may include at least one ofa reflective element (e.g., a reflection mirror) and at least onediffractive element (e.g., a diffractive optical element (“DOE”) or aholographic optical element (“HOE”)). The screen display portions 215 aand 215 b may guide light emitted from a display (e.g., the firstdisplay 205 and the second display 210) to the eyes of the user, usingthe at least one diffractive element or the reflective element includedin the screen display portions 215 a and 215 b.

According to various embodiments, the diffractive element may includethe input optical members 220 a and 220 b and/or an output opticalmember (e.g., the output optical member 340 of FIG. 3 ). In anembodiment, the input optical member 220 a, 220 b may refer to an inputgrating area, and the output optical member 340 may refer to an outputgrating area, for example. The input grating area may function as aninput terminal to diffract (or reflect) light output from the firstdisplay 205 and/or the second display 210 (e.g., a micro LED) totransmit the light to the screen display portions 215 a and 215 b. Theoutput grating area may function as an exit to diffract (or reflect)light transmitted to the display waveguide 350 and/or the eye trackingwaveguide 360 to the eye 301 of the user.

According to various embodiments, the reflective element may include atotal reflection optical element or a total reflection waveguide fortotal internal reflection (“TIR”). In an embodiment, total reflection,which is one of schemes of inducing light, may define an angle ofincidence such that light (e.g., a virtual image) entering through theinput grating area is completely or almost completely reflected from aportion (e.g., a predetermined surface) of the screen display portions215 a and 215 b, to completely or almost completely transmit the lightto the output grating area, for example.

The first transparent member 225 a and/or the second transparent member225 b may be formed as a glass plate, a plastic plate, or a polymer, andmay be transparently or translucently formed, for example. In anembodiment, the first transparent member 225 a may face the right eye ofthe user, and the second transparent member 225 b may face the left eyeof the user.

The lighting units 230 a and 230 b may be used differently according topositions in which the lighting units 230 a and 230 b are attached. Inan embodiment, the lighting units 230 a and 230 b may be attached arounda frame of the wearable electronic device 200, for example. The lightingunits 230 a and 230 b may be used as auxiliary devices for facilitatingeye-gaze detection when pupils are captured using the first eye trackingcamera 270 a and the second eye tracking camera 270 b. The lighting unit230 a, 230 b may use an infrared LED with a visible light wavelength oran infrared light wavelength.

In an alternative embodiment, the lighting unit 230 a, 230 b may beattached around a hinge (e.g., the first hinge 240 a and the secondhinge 240 b) that connects a temple (e.g., a temple 737 of FIG. 7 ) anda frame (e.g., a rim 910 of FIGS. 9A and 9B) of the wearable electronicdevice 200, or around a camera (e.g., the first recognition camera 265 aand the second recognition camera 265 b) disposed (e.g., mounted)adjacent to a bridge that connects the frame (e.g., the rim 910). In anembodiment, the first recognition camera 265 a and the secondrecognition camera 265 b may be global shutter (“GS”) cameras, forexample, but are not limited thereto.

When capturing is performed using a GS camera, the lighting units 230 aand 230 b may be used to supplement a surrounding brightness. In anembodiment, the lighting units 230 a and 230 b may be used in a darkenvironment or when it is not easy to detect a subject to be captureddue to reflected light and mixing of various light sources, for example.

In an alternative embodiment, the lighting units 230 a and 230 b may beomitted. The lighting units 230 a and 230 b may be replaced by infraredpixels included in the first display 205 and the second display 210. Inan embodiment, the lighting units 230 a and 230 b may be included in thewearable electronic device 200 to assist infrared pixels included in thefirst display 205 and the second display 210.

A PCB (e.g., the first PCB 235 a and the second PCB 235 b) may bedisposed in the temple (e.g., the temple 737) of the wearable electronicdevice 200, and may transmit an electric signal to each module (e.g.,camera, display, audio, or sensor modules) and another PCB through aflexible printed circuit board (“FPCB”). According to variousembodiments, at least one PCB may include the first PCB 235 a, thesecond PCB 235 b, and an interposer (not shown) disposed between thefirst PCB 235 a and the second PCB 235 b.

In an embodiment, a control circuit (not shown) for controllingcomponents of the wearable electronic device 200 other than the firstdisplay 205 and the second display 210 may be disposed on a PCB (e.g.,the first PCB 235 a and the second PCB 235 b). The control circuit maycontrol components other than the first display 205 and the seconddisplay 210 and perform an operation such as depth value estimation. Thecontrol circuit may include a communication circuit (e.g., thecommunication module 190 of FIG. 1 ) or a memory (e.g., the memory 130of FIG. 1 ). The control circuit may control the first display 205, thesecond display 210, and/or the other components.

The first hinge 240 a and/or the second hinge 240 b may correspond to aportion where the frame (e.g., the rim 910 of FIGS. 9A and 9B) and thetemple (e.g., the temple 737) of the wearable electronic device 200 arecoupled.

In an embodiment, the imaging camera 245 may be also referred to as a“high resolution (“HR”)” or a “photo video (“PV”)”, and may include ahigh-resolution camera. The imaging camera 245 may include a colorcamera having functions for obtaining a high-quality image, such as anautomatic focus (“AF”) function and an optical image stabilizer (“OIS”).The invention is not limited thereto, and the imaging camera 245 mayinclude a GS camera or a rolling shutter (“RS”) camera.

In an embodiment, a plurality of microphones (e.g., the first microphone250 a, the second microphone 250 b, and the third microphone 250 c) mayconvert an external acoustic signal into electrical audio data. Theelectrical audio data may be variously utilized according to a function(or an application being executed) being performed by the wearableelectronic device 200.

In an embodiment, a plurality of speakers (e.g., the first speaker 255 aand the second speaker 255 b) may output audio data that is receivedfrom a communication circuit (e.g., the communication module 190 of FIG.1 ) or stored in a memory (e.g., the memory 130 of FIG. 1 ).

In an embodiment, one or more batteries 260 may be included, and maysupply power to components constituting the wearable electronic device200.

In an embodiment, the first recognition camera 265 a and the secondrecognition camera 265 b may include cameras used for three degrees offreedom (“3DoF”) and six degrees of freedom (“6DoF”) head tracking, handdetection and tracking, and gesture and/or space recognition. In anembodiment, the first recognition camera 265 a and the secondrecognition camera 265 b may each include a GS camera to detect amovement of a head or a hand and track the movement, for example. In anembodiment, a stereo camera may be used for head tracking and spacerecognition, and accordingly two GS cameras with the same standard andperformance may be used, for example. An RS camera may be used to detecta quick hand movement and a minute movement of a finger and track amovement. In an embodiment, a GS camera having superior performance(e.g., image drag) in comparison to a camera may be mainly used,however, the invention is not limited thereto. According to variousembodiments, an RS camera may be used. The first recognition camera 265a and the second recognition camera 265 b may perform space recognitionfor 6DoF and a simultaneous localization and mapping (“SLAM”) functionthrough depth imaging. In addition, the first recognition camera 265 aand the second recognition camera 265 b may perform a user gesturerecognition function.

In an embodiment, at least one sensor (not shown), e.g., a gyro sensor,an acceleration sensor, a geomagnetic sensor, and/or a gesture sensor,the first recognition camera 265 a, and the second recognition camera265 b may perform at least one of head tracking for 6DoF, poseestimation and prediction, gesture and/or space recognition, and afunction of a SLAM through depth imaging.

In an embodiment, the first recognition camera 265 a and the secondrecognition camera 265 b may be classified and used as a camera for headtracking and a camera for hand tracking.

In an embodiment, the first eye tracking camera 270 a and the second eyetracking camera 270 b may detect and track pupils. The first eyetracking camera 270 a and the second eye tracking camera 270 b may beused to allow a center of a virtual image projected onto the wearableelectronic device 200 to be disposed based on a direction in which apupil of a user wearing the wearable electronic device 200 gazes. In anembodiment, as the first eye tracking camera 270 a and the second eyetracking camera 270 b, a GS camera may be mainly used to detect a pupiland track a fast pupil movement, for example. The first eye trackingcamera 270 a may be installed to correspond to the right eye of theuser, and the second eye tracking camera 270 b may be installed tocorrespond to the left eye of the user. Here, the first eye trackingcamera 270 a and the second eye tracking camera 270 b may have the samecamera performance and specifications, however, the invention is notlimited thereto. An operation of an eye tracking camera (e.g., the firsteye tracking camera 270 a and the second eye tracking camera 270 b) willbe described in more detail below with reference to FIG. 3 .

FIG. 3 is a diagram illustrating an embodiment of an operation of an eyetracking camera included in a wearable electronic device. FIG. 3illustrates a process in which an eye tracking camera 310 (e.g., thefirst eye tracking camera 270 a and the second eye tracking camera 270 bof FIG. 2 ) of a wearable electronic device 300 in an embodiment tracksthe eye 301 of the user, that is, a gaze of the user, using light (e.g.,infrared light) output from a display 320 (e.g., the first display 205and the second display 210 of FIG. 2 ).

The eye tracking camera (also indicated as ET camera in FIG. 3 ) 310 mayinclude the eye tracking sensor (also indicated as ET sensor in FIG. 3 )315. The eye tracking sensor 315 may be included inside the eye trackingcamera 310. The eye tracking sensor 315 may detect first reflected lightthat is generated by reflecting reflected infrared light 303 from theeye 301 of the user. The eye tracking camera 310 may track the eye 301of the user, that is, the gaze of the user, based on a detection resultof the eye tracking sensor 315.

The display 320 may include a plurality of visible light pixels and aplurality of infrared pixels. The visible light pixels may include R, G,and B pixels. However, the invention is not limited thereto, and thevisible light pixels may include various other color pixels. The visiblelight pixels may output visible light corresponding to a virtual objectimage. The infrared pixels may output infrared light. In an embodiment,the display 320 may include micro LEDs, or OLEDs, for example.

The wearable electronic device 300 may perform gaze tracking using theinfrared light output from the display 320. The projection lens 325(e.g., a projection lens 415 of FIG. 4 ) may be disposed between thedisplay 320 and an input optical member 330 (e.g., the input opticalmembers 220 a and 220 b of FIG. 2 ).

The infrared light output from the display 320 may be incident on theinput optical member 330 through the projection lens 325, and may beseparated into reflected infrared light 303 and transmitted infraredlight 305 by a half mirror (not shown) included in the input opticalmember 330.

The half mirror may be formed in the entire area or a partial area ofthe input optical member 330. When the half mirror is formed in theentire area of the input optical member 330, the input optical member330 may be also referred to as a “half mirror”. The half mirror may bedisposed in the input optical member 330 of the display waveguide 350.The half mirror may be disposed inside or below the input optical member330. The half mirror may include a grating structure.

The half mirror may output reflected infrared light and transmittedinfrared light in response to the infrared light output from the display320. The half mirror may include a grating structure. The gratingstructure may output reflected infrared light directed toward the eye301 of the user by reflecting a portion of the output infrared light, ormay output the reflected infrared light 303 toward the eye 301 of theuser through the output optical member 340 by passing through thedisplay waveguide 350. Also, the grating structure may output thetransmitted infrared light 305 by transmitting another portion of theoutput infrared light.

The reflected infrared light 303 may be output directly toward the eye301 of the user. The reflected infrared light 303 may be output towardthe eye 301 of the user through the output optical member 340 by passingthrough the display waveguide 350. The transmitted infrared light 305may be output toward the real world. The transmitted infrared light 305may be incident on the real object and may be partially reflected fromthe real object.

The display waveguide 350 and the eye tracking waveguide (also indicatedas ET waveguide in FIG. 3 ) 360 may be included in a transparent member370 (e.g., the first transparent member 225 a and the second transparentmember 225 b of FIG. 2 ). In an embodiment, the transparent member 370may be formed as a glass plate, a plastic plate, or a polymer, and maybe transparently or translucently formed, for example. The transparentmember 370 may face an eye of a user. In this case, a distance betweenthe transparent member 370 and the eye 301 may be also referred to as an“eye relief” 380.

The transparent member 370 may include the display waveguide 350 and theeye tracking waveguide 360. The transparent member 370 may include theinput optical member 330 and the output optical member 340. In addition,the transparent member 370 may include an eye tracking splitter (alsoindicated as ET tracking splitter in FIG. 3 ) 375 for splitting inputlight into multiple waveguides.

The display waveguide 350 is separated from the input optical member 330as shown in FIG. 3 , however, this is merely one of embodiments. Inanother embodiment, the input optical member 330 may be included in thedisplay waveguide 350.

In addition, the output optical member 340 is separated from the eyetracking waveguide 360, as shown in FIG. 3 , however, this is merely oneof embodiments. In another embodiment, the output optical member 340 maybe included in the eye tracking waveguide 360.

An optical waveguide (e.g., the display waveguide 350 and the eyetracking waveguide 360) may output a virtual object image by adjusting apath of visible light. Visible light and infrared light output from thedisplay 320 may be incident on the input optical member 330 through theprojection lens 325. Visible light among light incident on the inputoptical member 330 may be totally reflected through the displaywaveguide 350 to be guided to the output optical member 340. The visiblelight may be output from the output optical member 340 toward the eye301 of the user.

The wearable electronic device 300 may reflect or transmit the infraredlight output from the display 320 through the half mirror. In anembodiment, the wearable electronic device 300 may output the reflectedinfrared light 303 that is reflected by the half mirror (not shown)directly toward the eye 301 of the user, or may output the reflectedinfrared light 303 passing through the display waveguide 350 toward theeye 301 of the user. In an embodiment, the wearable electronic device300 may output the transmitted infrared light 305 passing through thehalf mirror toward the real object. A reflectivity and a transmittanceof the half mirror may be adjusted. In an embodiment, the half mirrormay have a reflectivity of about 30% (e.g., reflection toward eyes of auser) and a transmittance of about 70% (e.g., output toward a realobject) with respect to infrared light, for example. However, thereflectivity and the transmittance are merely some of embodiments andmay be adjusted in various ratios in other embodiments.

In an embodiment, the wearable electronic device 300 may output thereflected infrared light 303 toward eyes of the user through the halfmirror and the infrared pixels included in the display 320. Thereflected infrared light 303 may be reflected from the eye 301 of theuser, and the eye tracking sensor 315 may detect the reflected light.The display 320 including the infrared pixels, and the half mirrorincluded in the display waveguide 350 may be used instead of a separateinfrared light source for detecting a real object. Since the separateinfrared light source is not used, the wearable electronic device 300may be lightened and power consumption may be reduced. In addition, thedisplay 320 including the infrared pixels may function as an auxiliarylight source to increase an image quality of a stereo camera (e.g., thefirst recognition camera 265 a and the second recognition camera 265 bof FIG. 2 ) in a low-illuminance environment and increase an accuracy ofdepth information.

In an alternative embodiment, the wearable electronic device 300 mayoutput infrared light through the display 320 and detect light reflectedfrom the real object through a stereo camera (e.g., the firstrecognition camera 265 a and the second recognition camera 265 b of FIG.2 ). The wearable electronic device 300 may estimate a distance to thereal object based on a detection result. In an embodiment, the wearableelectronic device 300 may measure a depth value or use a time of flight(“ToF”) scheme to estimate the distance to the real object, for example.

The wearable electronic device 300 (e.g., the wearable electronic device200 of FIG. 2 ) may provide AR to a user. The wearable electronic device300 may provide an image representing the real world through thetransparent eye tracking waveguide 360, while transferring a virtualobject image output from the display 320 toward eyes of the user throughthe display waveguide 350.

The wearable electronic device 300 may include a head-mounted display(“HMD”), a face-mounted display (“FMD”), or a smart glass or a headsetthat provides extended reality such as AR, VR, or mixed reality, forexample, but is not limited thereto.

In an embodiment, the wearable electronic device 300 may output infraredlight using the display 320 including the infrared pixels. The wearableelectronic device 300 may track a gaze of a user, using the infraredlight output from the display 320. In addition, the wearable electronicdevice 300 may estimate a distance to a real object, using the infraredlight output from the display 320.

In an embodiment, the display 320 may include micro-reflection mirrors390.

FIG. 4 is a diagram illustrating an embodiment of operations of adisplay and screen display portions of a wearable electronic device.FIG. 4 illustrates an embodiment of a display 410 (e.g., the firstdisplay 205 and the second display 210 of FIG. 2 and the display 320 ofFIG. 3 ) and a screen display portion 450 (e.g., the screen displayportions 215 a and 215 b of FIG. 2 ) of a wearable electronic device(e.g., the wearable electronic device 200 of FIG. 2 and the wearableelectronic device 300 of FIG. 3 ). The display 410 may be also referredto as a “display module.”

The display 410 may output a virtual image (e.g., the virtual image 610of FIGS. 6A and 6B) to be used to evaluate deformation of the screendisplay portion 450. The virtual image may be a resolution evaluationchart such as a Photographic and Imaging Manufacturers Association(“PIMA”) chart or a circular zone plate chart, for example, but is notnecessarily limited thereto.

The display 410 may include a light source 411, a display device 413,and a projection lens 415.

In an embodiment, when the display device 413 is a digital lightprocessor (“DLP”) that implements high-precision display of an imageusing a DMD chip or an LCoS in which liquid crystal is installed on abackplane formed on a silicon wafer, for example, the light source 411may include a plurality of visible light pixels and a plurality ofinfrared pixels. The visible light pixels may include R, G, and Bpixels. However, the invention is not limited thereto, and the visiblelight pixels may include various other color pixels. The visible lightpixels may output visible light corresponding to a virtual object image.The infrared pixels may output infrared light.

In an embodiment, the display device 413 may include micro LEDs, orOLEDs, for example.

Light output through the light source 411 may be transmitted to an inputoptical member 430 through the display device 413 and the projectionlens 415. In an embodiment, the display device 413 may include aself-luminous device (e.g., micro LEDs, or OLEDs) that does not desirethe light source 411.

The virtual image 610 output through the display 410 may be transmittedto the screen display portion 450 through the projection lens 415 (e.g.,the projection lens 325 of FIG. 3 ) and the input optical member 430(e.g., the input optical members 220 a and 220 b of FIG. 2 and the inputoptical member 330 of FIG. 3 ). The input optical member 430 may referto an input grating area. The input optical member 430 may function asan input terminal to diffract (or reflect) light output from the display410. Light received from the input optical member 430 may be transmittedto an output optical member (e.g., the output optical member 340 of FIG.3 ) through a total reflection. The output optical member 340 may referto an “output grating area”. The output optical member 340 may functionas an exit to diffract (or reflect) light transferred through a totalreflection waveguide to eyes of a user.

The virtual image 610 transmitted to the screen display portion 450 maybe projected onto a flat surface or a portion (e.g., a portion 511 ofFIG. 5 ) of an interior of a case (e.g., a case 510 of FIG. 5 ) thatfunctions as a screen, through a focal lens (e.g., a focal lens 515 ofFIG. 5 ) in the case (e.g., the case 510) in which the wearableelectronic device 200, 300 is disposed (e.g., seated).

The wearable electronic device 200, 300 may capture the virtual image610 projected onto the portion 511 of the case 510, using eye trackingcameras for a left eye and a right eye (e.g., the first eye trackingcamera 270 a and the second eye tracking camera 270 b of FIG. 2 ) fixedto a rim (e.g., a rim 910 of FIGS. 9A and 9B) that encloses atransparent member 470 (e.g., the first transparent member 225 a and thesecond transparent member 225 b of FIG. 2 ). An image processor (e.g.,the processor 120 of FIG. 1 ) included in the wearable electronic device200, 300 may compare images captured by the eye tracking cameras (e.g.,the first eye tracking camera 270 a and the second eye tracking camera270 b of FIG. 2 ) to each other.

FIG. 5 is a diagram illustrating an embodiment of an adjustment deviceincluding a wearable electronic device and a case. FIG. 5 illustrates astructure of an adjustment device 500 for adjusting a deviation througha wearable electronic device 530 and the case 510.

The wearable electronic device 530 may be seated in the case 510. Thecase 510 may include a stator 513 and the focal lens 515.

The stator 513 may fix the wearable electronic device 530 in the case510. As shown in FIG. 5 , the stator 513 may fix a temple (e.g., thetemple 737 of FIG. 7 ) of the wearable electronic device 530, or a frame(e.g., the rim 910 of FIGS. 9A and 9B) of the wearable electronic device530. The stator 513 may fix at least a portion of the wearableelectronic device 530 to adjust a deviation.

The focal lens 515, instead of eyes of a user, may help output beams ofscreen display portions of the wearable electronic device 530 seated inthe case 510 form an image on the flat surface of the case 510 or theportion 511 of the case 510. More specifically, in the case 510, thefocal lens 515 may be disposed within an eye relief of the wearableelectronic device 530 fixed by the stator 513, to allow an image of avirtual image (e.g., the virtual image 610 of FIGS. 6A and 6B)transmitted to each of the screen display portions (e.g., the screendisplay portions 215 a and 215 b of FIG. 2 ) of the wearable electronicdevice 530 to be formed on the portion 511 of the case 510. In thiscase, the virtual image 610 may be output through displays 535 (e.g.,the first display 205 and the second display 210 of FIG. 2 ). Anembodiment of the virtual image 610 will be described in more detailbelow with reference to FIGS. 6A and 6B.

The portion 511 of the case 510 on which the image is formed may be aflat surface without a curvature to be used as a screen. Portions otherthan the portion 511 of the case 510 may be matt-coated,non-reflective-coated, or black-painted, to prevent unnecessary lightfrom being diffusely reflected and mixed into an eye tracking camera.

The wearable electronic device 530 may capture an image obtained byprojecting the virtual image 610 for measuring a deviation between thescreen display portions 215 a and 215 b onto the portion 511 of the case510 using the eye tracking cameras (e.g., the first eye tracking camera270 a and the second eye tracking camera 270 b of FIG. 2 ), and mayadjust the screen display portions 215 a and 215 b based on a comparisonresult of the captured images.

In addition, the wearable electronic device 530 may further include aprocessor (e.g., the processor 120 of FIG. 1 ) and a driving device(e.g., a driving device 813 of FIGS. 8A and 8B, driving devices 920 and930 of FIGS. 9B and 9C, and a driving device 1010 of FIG. 10 ). Theprocessor 120 may process images captured by the first eye trackingcamera 270 a and the second eye tracking camera 270 b and calculate thedeviation between the screen display portions 215 a and 215 b. Thedriving device 813, 920, 930, 1010 may adjust the screen displayportions 215 a and 215 b based on the deviation calculated by theprocessor 120. The wearable electronic device 530 may control the screendisplay portions 215 a and 215 b by controlling the driving device 813,920, 930, 1010 such that the deviation between the screen displayportions 215 a and 215 b may be minimized.

The processor 120 may calculate a number of lines of the virtual image610 for measuring the deviation and a width of each of the lines fromeach of the captured images. The processor 120 may determine whether toadjust the deviation between the screen display portions 215 a and 215 bbased on a result of comparing the number and width of the lines of thevirtual image 610 to a predetermined threshold. The processor mayperform calibration on the left and right eyes of the screen displayportions 215 a and 215 b based on a determination to adjust thedeviation between the screen display portions 215 a and 215 b.

In an embodiment, an image of the virtual image 610 displayed on thescreen display portions 215 a and 215 b of the wearable electronicdevice 530 may be allowed to be formed on the portion 511 of the case510 without a curvature, using the focal lens 515 disposed within an eyerelief (e.g., the eye relief 380 of FIG. 3 ) of the wearable electronicdevice 530 seated based on a position of the stator 513 in the case 510,so that a deviation between the left eye and the right eye may bemeasured even though there is no separate checkerboard chart.

In addition, in an embodiment, a degree (e.g., a deviation) to which thescreen display portions 215 a and 215 b deviate may be measured byimages captured using the first eye tracking camera 270 a and the secondeye tracking camera 270 b included in the wearable electronic device530, even though there is no separate capturing device, and the screendisplay portions 215 a and 215 b may be adjusted to minimize thedeviation.

When it is determined that the wearable electronic device 530 is seatedin the case 510, the wearable electronic device 530 may transmit thevirtual image 610 to adjust the deviation. Whether the wearableelectronic device 530 is seated in the case 510 may be determined basedon whether charging of the wearable electronic device 530 seated in thecase 510 starts, whether a sensor for sensing a hole (e.g., a hole 815of FIG. 8A) of the driving device 813 senses whether a shaft 831 of awearable electronic device 830 of FIG. 8A is fastened or coupled to thehole 815, or may be determined through power line communication (“PLC”)or short-distance communication between the case 510 and the wearableelectronic device 530, for example.

When the wearable electronic device 530 is seated in the case 510, theprocessor 120 of the wearable electronic device 530 may transmit avirtual image for adjusting a deviation to the portion 511 of the case510. In an embodiment, deviation adjustment may be performed every timethe wearable electronic device 530 is seated in the case 510, or may beperformed at regular intervals, for example.

In an embodiment, in addition to adjusting of the deviation between thescreen display portions 215 a and 215 b, the wearable electronic device530 may pre-store reference image information used to determine whetherto adjust a deviation and may independently correct a left image and aright image of the screen display portions 215 a and 215 b based on thereference image information.

Although an example in which the temple (e.g., the temple 737) of thewearable electronic device 530 is unfolded is described above withreference to FIG. 5 , the invention is not limited thereto. In anembodiment, the temple (e.g., the temple 737) of the wearable electronicdevice 530 may be folded and the wearable electronic device 530 may beseated, thereby reducing the size and weight of the case 510, forexample. An example in which the wearable electronic device 530 with thefolded temple (e.g., the temple 737) is seated in the case 510 will bedescribed in more detail below with reference to FIG. 7 .

FIG. 6A is a diagram illustrating an embodiment of a virtual imageprojected by an adjustment device, and 6B is an enlarged view of aportion indicated by a dot-dash line in FIG. 6A. FIG. 6A illustrates anembodiment of the virtual image 610 projected onto a portion 605 (e.g.,the portion 511 of FIG. 5 , and a portion 711 of FIG. 7 ) of an interiorof a case. In an embodiment, the virtual image 610 may be a PIMA chart,for example, as shown in FIGS. 6A and 6B. In an alternative embodiment,the virtual image 610 may be a circular zone plate chart, or variouscharts available for resolution evaluation. Hereinafter, the PIMA chartwill be mainly described for convenience of description, however, theinvention is not limited thereto.

The PIMA chart may be used to measure a resolution. In the virtual image610, vertical black lines 611 and white lines 613 may correspond to PIMAlines. Also, a gap between the black lines 611 and the white lines 613may correspond to a width of a PIMA line.

An image processor (e.g., the processor 120 of FIG. 1 ) of a wearableelectronic device 630 may calculate a number of PIMA lines and a widthof each of the PIMA lines from images captured by eye tracking cameras(e.g., the first eye tracking camera 270 a and the second eye trackingcamera 270 b of FIG. 2 ), which will be described in more detail belowwith reference to FIG. 12 . Here, the width of the PIMA line maycorrespond to a number of pixels, that is, a minimum number of pixels tobe recognized as the black lines 611 and the white lines 613. Also, thenumber of PIMA lines may correspond to a number of black lines 611 andwhite lines 613 and may refer to a frequency. The number of PIMA linesand the width of the PIMA lines may be used to determine a resolutionincluding a minimum number of pixels to be recognized for eachfrequency, which may be a criterion for determining whether to alignscreen display portions of the wearable electronic device 630.

The wearable electronic device 630 may determine whether resultsobtained by calculating the number of PIMA lines and the width of eachof the PIMA lines exceed a predetermined threshold, by comparing theresults to each other. Here, the determining of whether the resultsexceed the predetermined threshold may indicate how many lines in pairsof black lines 611 and white lines 613 may be read, that is, may be acriterion for determining a resolution. When a comparison resultobtained by calculating the number and width of PIMA lines in the imagesrespectively captured by the first eye tracking camera 270 a and thesecond eye tracking camera 270 b exceeds a predetermined threshold, thewearable electronic device 630 may align the screen display portions(e.g., the screen display portions 215 a and 215 b of FIG. 2 ).

FIG. 7 is a diagram illustrating another embodiment of a structure of anadjustment device including a wearable electronic device and a case.FIG. 7 illustrates a wearable electronic device with folded temples 737seated in a case (e.g., the case 510 of FIG. 5 ).

As shown in FIG. 7 , in a state in which a hinge 731 (e.g., the firsthinge 240 a and the second hinge 240 b of FIG. 2 ) of a wearableelectronic device 730 is folded and the temples 737 are gathered, thewearable electronic device 730 is seated in a case 710. Here, imagecapturing may be hindered due to the temples 737. In an embodiment, afocal lens 715 (e.g., the focal lens 515 of FIG. 5 ) may be moved towardto screen display portions (e.g., the screen display portions 215 a and215 b of FIG. 2 ) of the wearable electronic device 730, in comparisonto the embodiment of FIG. 5 . In addition, a portion 711 on which animage of a virtual image (e.g., the virtual image 610 of FIGS. 6A and6B) output from each of the screen display portions (e.g., the screendisplay portions 215 a and 215 b of FIG. 2 ) of the wearable electronicdevice 730 may be disposed between the folded temple 737 and the focallens 715. Here, the virtual image 610 may be output through displays 735(e.g., the first display 205 and the second display 210 of FIG. 2 ). Inan embodiment, the focal lens 715 may include a lens having a shortfocal distance such as a high refractive lens, but is not limitedthereto.

In an embodiment, the portion 511, 711 on which the image of the virtualimage 610 is formed through the focal lens 515, 715 may be fixed at aposition in the case 510, 710. In an alternative embodiment, a focalpoint may be set by a user moving the focal lens 515, 715 or one of theportions 511 and 711 on which images are formed.

FIG. 8A is a diagram illustrating an embodiment of a method of adjustingscreen display portions of a wearable electronic device using a drivingdevice included in a case, and FIG. 8B is an enlarged view of a portionindicated by a dotted line in FIG. 8A. FIG. 8A illustrates an embodimentof a stator (e.g., the stator 513 of FIG. 5 ) of a case 810 furtherincluding the driving device 813 for calibration. The driving device 813may be included in the stator 513 of the case 810 or may correspond tothe stator 513 itself. In an embodiment, the driving device 813 may be amotor or a magnet, for example, but is not limited thereto. One drivingdevice 813, or a plurality of driving devices 813 may be provided. In anembodiment, the driving device 813 may include a driving device for aright eye and a driving device for a left eye.

The wearable electronic device 830 may include the shaft 831 coupled tothe driving device 813. In an embodiment, the shaft 831 may be providedin each of left and right outer rims (e.g., the rim 910 of FIGS. 9A and9B) of the wearable electronic device 830 or each of left and righttemples (e.g., the temples 737 of FIG. 7 ), and may include a gear unit,for example. The shaft 831 may be accommodated in the rim 910 or thetemple 737. When calibration is performed, the shaft 831 may protrudeout of the rim 910 or the temple 737 in the same manner as a pushbutton, and may be fastened to the hole 815 in the driving device 813.In an embodiment, the wearable electronic device 830 may further includea sensor for sensing the hole 815. When the sensor for sensing the hole815 detects that the shaft 831 of the wearable electronic device 830 isfastened to the hole 815 of the driving device 813, the wearableelectronic device 830 may be determined to be seated in the case 810.

When calibration is performed, the shaft 831 may protrude from the rim910 or the temple 737. When the calibration is completed, the shaft 831may be accommodated back in the frame (e.g., rim 910) or temple 737.

In an embodiment, the shaft 831 may be coupled to the hole 815 of thedriving device 813, for example. When the driving device 813 coupled tothe shaft 831 horizontally or vertically moves screen display portions(e.g., the screen display portions 215 a and 215 b of FIG. 2 ) of thewearable electronic device 830, the screen display portions 215 a and215 b may be adjusted or aligned.

The driving device 813 of the stator may be coupled to the shaft 831 ofthe wearable electronic device 830 to adjust the screen display portions215 a and 215 b based on a deviation between the screen display portions215 a and 215 b.

In an embodiment, when it is determined that the screen display portions(e.g., the screen display portions 215 a and 215 b of FIG. 2 ) need tobe aligned based on images captured by each of eye tracking cameras(e.g., the first eye tracking camera 270 a and the second eye trackingcamera 270 b), the wearable electronic device 830 may transmitinformation to the case 810 including the driving device 813 through acommunication module (e.g., the communication module 190 of FIG. 1 ). Inan embodiment, the case 810 may control the driving device 813 based onthe information received from the wearable electronic device 830 toadjust the screen display portions 215 a and 215 b.

FIGS. 9A to 9C are diagrams illustrating an embodiment of a method ofadjusting screen display portions of a wearable electronic device usinga driving device included in the wearable electronic device. FIG. 9Billustrates a magnet 920, a coil 930, and a spring 940 that fix screendisplay portions 901 (e.g., the screen display portions 215 a and 215 bof FIG. 2 ) and transparent member(s) 903 (e.g., the first transparentmember 225 a and the second transparent member 225 b of FIG. 2 ) of awearable electronic device 900.

The wearable electronic device 900 may further include driving devices920 and 930 provided in each of the screen display portions 901. Thewearable electronic device 900 may adjust, using the driving devices 920and 930, the screen display portions 901 in directions with six degreesof freedom (“6DOF”) (e.g., X, Y, Z, yaw, roll and pitch directions)based on a deviation between the surface display portions 901.

Here, each of the screen display portions 901 may be fixed to the rims910 of the wearable electronic device 900 for the right eye and the lefteye, together with the transparent member 903, by at least one of themagnet 920, the coil 930 and the spring 940. An adjustment device (e.g.,the adjustment device 500 of FIG. 5 ) may adjust the screen displayportions 901 of the wearable electronic device 900 based on thedeviation, by the magnet 920 and the coil 930 that fix each of thescreen display portions 901. In an embodiment, a processor (e.g., theprocessor 120 of FIG. 1 ) of the wearable electronic device 900 mayadjust the magnet 920 and the coil 930 that fix each of the screendisplay portions 901, based on images captured by each of eye trackingcameras (e.g., the first eye tracking camera 270 a and the second eyetracking camera 270 b of FIG. 2 ) for the left eye and the right eye,for example.

Examples of operations of the magnet 920, the coil 930 and the spring940 will be further described below. The adjustment device 500 maycontrol the screen display portions 901 of the wearable electronicdevice 900 in directions with 6DOF, based on a principle that a force isgenerated in a direction perpendicular to a magnetic field when acurrent flows in the coil 930 around the magnet 920.

In an embodiment, the wearable electronic device 900 may further includedisplays (e.g., the first display 205 and the second display 210 of FIG.2 ) including micro-reflection mirrors (e.g., micro-reflection mirrors390) and projection lenses (e.g., the projection lens 325 of FIG. 3 ).Here, the wearable electronic device 900 may adjust focal distances ofthe screen display portions 901 based on the deviation between thescreen display portions 901 by adjusting at least one of an outputdirection and a view angle of light of projection lenses 325, using themicro-reflection mirrors (e.g., micro-reflection mirrors 390).

FIG. 10 is a diagram illustrating an embodiment of a method of adjustingscreen display portions of a wearable electronic device using a drivingdevice included in the wearable electronic device. FIG. 10 illustratesthe driving device 1010 that is attached to a rim (e.g., the rim 910 ofFIGS. 9A and 9B) of a wearable electronic device (e.g., the wearableelectronic device 200 of FIG. 2 , the wearable electronic device 300 ofFIG. 3 , the wearable electronic device 530 of FIG. 5 , the wearableelectronic device 630 of FIG. 6A, the wearable electronic device 730 ofFIG. 7 , the wearable electronic device 800 of FIG. 8A, and the wearableelectronic device 900 of FIG. 9A) in an embodiment to adjust screendisplay portions 1001 (e.g., the screen display portions 215 a and 215 bof FIG. 2 , the screen display portion 450 of FIG. 4 , and the screendisplay portions 901 of FIGS. 9A to 9C).

An adjustment device (e.g., the adjustment device 500 of FIG. 5 ) mayadjust the screen display portions 1001 by driving the driving device1010 of the wearable electronic device 200, 300, 530, 630, 730, 800,900. The driving device(s) 1010 may be installed on a left rim and aright rim (e.g., the rims 910 of FIGS. 9A and 9B) of the wearableelectronic device 200, 300, 530, 630, 730, 800, 900. The adjustmentdevice 500 may be adjusted by the driving device(s) 1010 based on adeviation between the screen display portions 1001 by the drivingdevice(s) 1010 installed on the rims 910 of the wearable electronicdevice 200, 300, 530, 630, 730, 800, 900.

The driving device(s) 1010 may include a motor 1015, and a second gearunit 1013 that transmits a force driven by the motor 1015.

A first gear unit 1030 provided on one side of the transparent member1003 including the screen display portion 1001, and the second gear unit1013 of the driving device 1010 may be coupled to each other, and movedhorizontally by the motor 1015, to adjust the screen display portion1001.

FIG. 11 is a flowchart illustrating an embodiment of a method ofoperating an adjustment device. In the following examples, operationsmay be performed sequentially, but not necessarily performedsequentially. In an embodiment, the order of the operations may bechanged and at least two of the operations may be performed in parallel,for example.

FIG. 11 illustrates a process in which an adjustment device in anembodiment adjusts screen display portions for a left eye and a righteye, through operations 1110 to 1140. The adjustment device (e.g., theadjustment device 500 of FIG. 5 ) may include a case (e.g., the case 510of FIG. 5 , the case 710 of FIG. 7 , and the case 810 of FIG. 8A) inwhich a wearable electronic device (e.g., the wearable electronic device200 of FIG. 2 , the wearable electronic device 300 of FIG. 3 , thewearable electronic device 530 of FIG. 5 , the wearable electronicdevice 630 of FIG. 6A, the wearable electronic device 730 of FIG. 7 ,the wearable electronic device 800 of FIG. 8A, and the wearableelectronic device 900 of FIG. 9A) that includes screen display portionsfor a left eye and a right eye (e.g., the screen display portions 215 aand 215 b of FIG. 2 , the screen display portion 450 of FIG. 4 , and thescreen display portions 901 of FIGS. 9A to 9C) and eye tracking camerasfor the left eye and the right eye (e.g., the first eye tracking camera270 a and the second eye tracking camera 270 b of FIG. 2 ) is seated.

In operation 1110, the adjustment device 500 may transmit a virtualimage (e.g., the virtual image 610 of FIGS. 6A and 6B) for measuring adeviation between the screen display portions (e.g., the screen displayportions 215 a and 215 b of FIG. 2 , the screen display portion 450 ofFIG. 4 , and the screen display portions 901 of FIGS. 9A to 9C) to thescreen display portions 215 a, 215 b, 450, 901. The adjustment device500 may output the virtual image 610 through displays for the left eyeand the right eye (e.g., the first display 205 and the second display210 of FIG. 2 , the display 320 of FIG. 3 , the display 410 of FIG. 4 ,the display 535 of FIG. 5 , and the display 735 of FIG. 7 ) of thewearable electronic device 200, 300, 530, 630, 730, 800, 900, totransmit the virtual image 610 to the screen display portions 215 a, 215b, 450, 901. In addition, the adjustment device 500 may transmit thevirtual image 610 to the screen display portions 215 a, 215 b, 450, 901through a projection lens (e.g., the projection lens 325 of FIG. 3 , andthe projection lens 415 of FIG. 4 ) and an input optical member (e.g.,the input optical members 220 a and 220 b of FIG. 2 , the input opticalmember 330 of FIG. 3 , and the input optical member 430 of FIG. 4 ) ofthe wearable electronic device 200, 300, 530, 630, 730, 800, 900. Here,the virtual image 610 may include at least one of a PIMA chart and acircular zone plate chart for resolution evaluation.

In operation 1120, the adjustment device 500 may project the virtualimage 610 onto a portion (e.g., the portion 511 of FIG. 5 , the portion605 of FIG. 6A, and the portion 711 of FIG. 7 ) of the case 510, 710,810 through a focal lens (e.g., the focal lens 515 of FIG. 5 , and thefocal lens 715 of FIG. 7 ) disposed in the case 510, 710, 810.

In operation 1130, the adjustment device 500 may capture virtual images610 projected onto the portions 511, 605, and 711 of the cases 510, 710,and 810 in operation 1120, using the first eye tracking camera 270 a andthe second eye tracking camera 270 b.

In operation 1140, the adjustment device 500 may adjust the screendisplay portions 215 a, 215 b, 450, 901 based on a comparison result ofthe images captured in operation 1130. The adjustment device 500 mayprocess the images captured in operation 1130 to calculate a deviationbetween the screen display portions 215 a, 215 b, 450, 901. Theadjustment device 500 may adjust the screen display portions 215 a, 215b, 450, 901 based on the deviation. In an embodiment, the adjustmentdevice 500 may adjust the screen display portions 215 a, 215 b, 450, 901so that a deviation between the left eye and the right eye may beminimized, for example.

In operation 1140, the adjustment device 500 may calculate a number ofPIMA lines displayed in each of the captured images, and a width of eachof the PIMA lines, for example. The adjustment device 500 may calculatethe deviation between the screen display portions 215 a, 215 b, 450, 901based on a result of comparing the number and width of the PIMA lines toa threshold. The adjustment device 500 may determine whether to adjustthe deviation between the screen display portions 215 a, 215 b, 450,901, based on the result of the comparing. When a difference in a numberand width of lines of the virtual image 610 between the captured imagesexceeds a predetermined threshold, the adjustment device 500 maydetermine to adjust the deviation between the left eye and the righteye. The adjustment device 500 may perform calibration on the screendisplay portions 215 a, 215 b, 450, 901 based on a determination as towhether to adjust the deviation.

In an embodiment, the adjustment device 500 may adjust the screendisplay portions 215 a, 215 b, 450, 901 based on the deviation, usingthe shaft 831 provided in each of a left temple and a right temple(e.g., the temple 737 of FIG. 7 ) or a rim (e.g., the rim 910 of FIGS.9A and 9B) of the wearable electronic device 200, 300, 530, 630, 730,800, 900, and using a driving device (e.g., the driving device 813 ofFIGS. 8A and 8B, the driving devices 920 and 930 of FIGS. 9B and 9C, andthe driving device 1010 of FIG. 10 ) installed on a stator (e.g., thestator 513 of FIG. 5 , and the stator 713 of FIG. 7 ) of the case 510,710, 810, as shown in FIG. 8A. In another embodiment, the adjustmentdevice 500 may adjust the screen display portions 215 a, 215 b, 450, 901in directions with 6DOF, based on the deviation, using the drivingdevice 920, 930, 1010 included in each of the screen display portions215 a, 215 b, 450, 901, as shown in FIGS. 9A to 10 .

In an embodiment, the wearable electronic device 200, 300, 530, 630,730, 800, 900 may further include the display 205, 210, 320, 410, 535,735 that includes micro-reflection mirrors (e.g., micro-reflectionmirrors 390) (e.g., micro electro mechanical system (“MEMS”) mirrors)and projection lenses 325 and 415. Here, the adjustment device 500 mayadjust at least one of an output direction and a view angle of light ofthe projection lenses 325 and 415 using the micro-reflection mirrors(e.g., micro-reflection mirrors 390) to adjust focal distances of thescreen display portions 215 a, 215 b, 450, 901 in operation 1140.

In an alternative embodiment, the display 205, 210, 320, 410, 535, 735of the wearable electronic device 200, 300, 530, 630, 730, 800, 900 mayinclude a display (not shown) adjustable to a multifocal plane. Here,the display adjustable to the multifocal plane may be a phase-modulatedmicro display (not shown), for example, but is not necessarily limitedthereto.

In addition, in operation 1140, the adjustment device 500 may adjust themultifocal plane through phase modulation of the display 205, 210, 320,410, 535, 735 to adjust the focal distances of the screen displayportions 215 a, 215 b, 450, 901 based on the deviation. In anembodiment, the adjustment device 500 may adjust the multifocal plane tominimize the deviation between the screen display portions 215 a, 215 b,450, 901 in the display 205, 210, 320, 410, 535, 735, for example.

FIG. 12 is a flowchart illustrating another embodiment of a method ofoperating an adjustment device. In the following examples, operationsmay be performed sequentially, but not necessarily performedsequentially. In an embodiment, the order of the operations may bechanged and at least two of the operations may be performed in parallel,for example.

FIG. 12 illustrates a process in which an adjustment device (e.g., theadjustment device 500 of FIG. 5 ) in an embodiment adjusts screendisplay portions (e.g., the screen display portions 215 a and 215 b ofFIG. 2 , the screen display portion 450 of FIG. 4 , the screen displayportions 901 of FIGS. 9A to 9C, and the screen display portions 1001 ofFIG. 10 ) for a left eye and a right eye, through operations 1205 to1270.

In operation 1205, the adjustment device 500 may capture virtual images610 (e.g., PIMA charts) projected onto a portion (e.g., the portion 511of FIG. 5 , the portion 605 of FIG. 6A, and the portion 711 of FIG. 7 )of a case (e.g., the case 510 of FIG. 5 , the case 710 of FIG. 7 , andthe case 810 of FIG. 8A), in which a wearable electronic device (e.g.,the wearable electronic device 200 of FIG. 2 , the wearable electronicdevice 300 of FIG. 3 , the wearable electronic device 530 of FIG. 5 ,the wearable electronic device 630 of FIG. 6A, the wearable electronicdevice 730 of FIG. 7 , the wearable electronic device 800 of FIG. 8A,and the wearable electronic device 900 of FIG. 9A) is seated, by eyetracking cameras for the left eye and the right eye (e.g., the first eyetracking camera 270 a and the second eye tracking camera 270 b of FIG. 2) of the wearable electronic device 200, 300, 530, 630, 730, 800, 900.

In operation 1210, the adjustment device 500 may scan a central portionand a neighboring portion of the virtual images 610 captured inoperation 1205 and may calculate (or obtain) a number of PIMA lines anda width of each of the PIMA lines in the central portion and neighboringportion of the virtual images 610, using an image processor (e.g., theprocessor 120 of FIG. 1 ) of the wearable electronic device 200, 300,530, 630, 730, 800, 900.

In operation 1215, the adjustment device 500 may determine whether anumber of PIMA lines corresponding to the left eye or a width of each ofthe PIMA lines is greater than or equal to a threshold, using the imageprocessor 120 of the wearable electronic device 200, 300, 530, 630, 730,800, 900. When it is determined in operation 1215 that the number ofPIMA lines corresponding to the left eye is greater than or equal to thethreshold, the adjustment device 500 may perform calibration on thescreen display portions 215 a, 215 b, 450, 901 for the left eye, usingthe image processor 120 of the wearable electronic device 200, 300, 530,630, 730, 800, 900 in operation 1220. Determining that the number ofPIMA lines is greater than or equal to the threshold in operation 1215may indicate that the screen display portions 215 a, 215 b, 450, 901 forthe left eye are significantly misaligned, and accordingly theadjustment device 500 may adjust the screen display portions 215 a, 215b, 450, 901 for the left eye to adjust a focus through calibration. Inan embodiment, the adjustment device 500 may perform calibration basedon the method of FIG. 8A, 9A, or 10, for example.

When it is determined in operation 1215 that the number of PIMA lines orthe width of the PIMA lines is less than the threshold, the adjustmentdevice 500 may determine whether a number of PIMA lines corresponding tothe right eye or a width of each of the PIMA lines is greater than orequal to the threshold, using the image processor 120 of the wearableelectronic device 200, 300, 530, 630, 730, 800, 900 in operation 1225.When it is determined in operation 1225 that the number of PIMA linescorresponding to the right eye is greater than or equal to thethreshold, the adjustment device 500 may perform calibration on thescreen display portions 215 a, 215 b, 450, 901 for the right eye, usingthe image processor 120 of the wearable electronic device 200, 300, 530,630, 730, 800, 900 in operation 1230.

In operation 1235, the adjustment device 500 may determine whether thecalibrations on the screen display portions 215 a, 215 b, 450, 901 forthe left eye and the right eye are completed, using the image processor120 of the wearable electronic device 200, 300, 530, 630, 730, 800, 900.When it is determined that the calibrations are not completed, theadjustment device 500 may allow the calibrations to be performed on thescreen display portions 215 a, 215 b, 450, and 901 for the left eye andthe right eye through operation 1215 or 1225. When it is determined inoperation 1235 that the calibrations are completed, the adjustmentdevice 500 may evaluate a difference in the number of PIMA lines of thescreen display portions 215 a, 215 b, 450, 901 for the left eye and theright eye, that is, a deviation between the screen display portions 215a, 215 b, 450, 901.

In operation 1240, the adjustment device 500 may determine whether thedeviation between the screen display portions 215 a, 215 b, 450, 901 isgreater than or equal to a threshold, using the image processor 120 ofthe wearable electronic device 200, 300, 530, 630, 730, 800, 900.

When it is determined in operation 1240 that the deviation is greaterthan or equal to the threshold, the adjustment device 500 may determinewhether the deviation between the screen display portions 215 a, 215 b,450, 901 is greater than or equal to a negative number, using the imageprocessor 120 of the wearable electronic device 200, 300, 530, 630, 730,800, 900 in operation 1245. When it is determined in operation 1245 thatthe deviation between the screen display portions 215 a, 215 b, 450, 901is greater than or equal to the negative number, the adjustment device500 may perform calibration in a reverse direction of a difference valuecorresponding to the deviation, using the image processor 120 of thewearable electronic device 200, 300, 530, 630, 730, 800, 900 inoperation 1250. Here, the adjustment device 500 may perform calibrationon the screen display portions 215 a, 215 b, 450, 901 for the right eyethrough operations 1225 and 1230.

When it is determined in operation 1245 that the deviation between thescreen display portions 215 a, 215 b, 450, 901 is less than the negativenumber, the adjustment device 500 may perform calibration in a reversedirection of a difference value corresponding to the deviation, usingthe image processor 120 of the wearable electronic device 200, 300, 530,630, 730, 800, 900 in operation 1255. Here, the adjustment device 500may perform calibration on the screen display portions 215 a, 215 b,450, 901 for the left eye through operations 1215 and 1220.

When it is determined in operation 1240 that the deviation is less thanthe threshold, the adjustment device 500 may finally terminate thecalibration in operation 1260. In operation 1260, the adjustment device500 may fix a screw of the stator 513, 713 of the case 510, 710, 810 orthe screen display portions 215 a, 215 b, 450, 901 of the wearableelectronic device 200, 300, 530, 630, 730, 800, 900 based on calibrationof a current state, may perform bonding, and may terminate thecalibration.

FIG. 13 is a flowchart illustrating another embodiment of a method ofoperating an adjustment device. In the following examples, operationsmay be performed sequentially, but not necessarily performedsequentially. In an embodiment, the order of the operations may bechanged and at least two of the operations may be performed in parallel,for example.

FIG. 13 illustrates a process in which an adjustment device (e.g., theadjustment device 500 of FIG. 5 ) in an embodiment adjusts a multiplefocal planes of the wearable electronic device, through operations 1310to 1330.

In operation 1310, the adjustment device 500 may provide displays of awearable electronic device including a display adjustable to amultifocal plane.

In operation 1320, the adjustment device 500 may adjust the multifocalplane through phase modulation of the displays provided in the operation1310.

In operation 1330, the adjustment device 500 may adjust focal distancesof the screen display portions based on a deviation adjusted throughphase modulation in operation 1320. In an embodiment, an adjustmentdevice 500 may include a wearable electronic device 200, 300, 530, 630,730, 800, 900 and a case 510, 710, 810 in which the wearable electronicdevice 200, 300, 530, 630, 730, 800, 900 is seated. The wearableelectronic device 200, 300, 530, 630, 730, 800, 900 may include displays205, 210, 320, 410, 535, 735 which display virtual images 610 for a lefteye and a right eye of a user, screen display portions 215 a, 215 b,450, 901 which transmit light sources generated in the displays 205,210, 320, 410, 535, 735 to the left eye and the right eye, and eyetracking cameras for the left eye and the right eye (e.g., the first eyetracking camera 270 a and the second eye tracking camera 270 b). Thecase 510, 710, 810 may include a stator 513, 713 which fixes thewearable electronic device 200, 300, 530, 630, 730, 800, 900, and afocal lens which is disposed within an eye relief of the fixed wearableelectronic device 200, 300, 530, 630, 730, 800, 900 and allows images ofthe virtual images 610 output from the screen display portions 215 a,215 b, 450, 901 of the wearable electronic device 200, 300, 530, 630,730, 800, 900 to be formed on a portion 511, 605, 711 of an interior ofthe case 510, 710, 810.

In an embodiment, the portion 511, 605, 711 of the case 510, 710, 810may include a flat surface without a curvature, and portions other thanthe portion 511, 605, 711 of the case 510, 710, 810 may be matt-coatedor black-painted.

In an embodiment, the wearable electronic device 200, 300, 530, 630,730, 800, 900 may capture images obtained by projecting the virtualimages 610 for measuring a deviation between the screen display portions215 a, 215 b, 450, 901 onto the portion 511, 605, 711 of the case 510,710, 810, using the first eye tracking camera 270 a and the second eyetracking camera 270 b, and may adjust the screen display portions 215 a,215 b, 450, 901 based on a comparison result of the captured images.

The wearable electronic device 200, 300, 530, 630, 730, 800, 900 mayfurther include a processor 120 which performs image processing on thecaptured images and calculates the deviation between the screen displayportions 215 a, 215 b, 450, 901, and a driving device 813, 920, 930, and1010 which adjusts the screen display portions 215 a, 215 b, 450, 901based on the deviation. The processor 120 may calculate a number oflines of a virtual image 610 for measuring a deviation and a width ofeach of the lines from each of the captured images, determine whether toadjust the deviation between the screen display portions 215 a, 215 b,450, 901 based on a result of comparing the number and the width of thelines of the virtual image 610 to a predetermined threshold, and performcalibration on the left eye and the right eye of the screen displayportions 215 a, 215 b, 450, 901 based on the determining.

In an embodiment, a shaft 831 may be provided in each of left and rightouter rims 910, or each of left and right temples (e.g., the temples 737of FIG. 7 ) of the wearable electronic device 200, 300, 530, 630, 730,800, 900. The stator 513, 713 may further include a driving device 813,920, 930, 1010 coupled to the shaft 831 to adjust the screen displayportions 215 a, 215 b, 450, 901.

In an embodiment, the wearable electronic device 200, 300, 530, 630,730, 800, 900 may further include a driving device 813, 920, 930, 1010provided in each of the screen display portions 215 a, 215 b, 450, 901,and may adjust the screen display portions 215 a, 215 b, 450, 901 basedon a deviation between the screen display portions 215 a, 215 b, 450,901 using the driving device 813, 920, 930, 1010.

In an embodiment, the screen display portions 215 a, 215 b, 450, 901 maybe fixed to a left rim and a right rim (e.g., rims 910) of the wearableelectronic device 200, 300, 530, 630, 730, 800, 900, by at least one ofa magnet 920, a coil 930, and a spring 940. Each of the screen displayportions 215 a, 215 b, 450, 901 may be adjusted based on the deviationby at least one of the magnet 920, the coil 830, and the spring 940.

In an embodiment, driving devices 813, 920, 930, 1010 may be installedon a left rim and a right rim (e.g., rims 910) of the wearableelectronic device 200, 300, 530, 630, 730, 800, 900, and the screendisplay portions 215 a, 215 b, 450, 901 may be adjusted by the drivingdevice 813, 920, 930, 1010 based on the deviation.

In an embodiment, the displays 205, 210, 320, 410, 535, 735 may furtherinclude micro-reflection mirrors (e.g., micro-reflection mirrors 390)and projection lenses 325 and 415. The wearable electronic device 200,300, 530, 630, 730, 800, 900 may adjust at least one of an outputdirection and a view angle of light of the projection lenses 325 and 415using the micro-reflection mirrors (e.g., micro-reflection mirrors 390),and adjust focal distances of the screen display portions 215 a, 215 b,450, 901 based on the deviation.

In an embodiment, the displays 205, 210, 320, 410, 535, 735 may includea display (not shown) adjustable to a multifocal plane. The wearableelectronic device 200, 300, 530, 630, 730, 800, 900 may adjust themultifocal plane through phase modulation of the displays 205, 210, 320,410, 535, 735, and adjust focal distances of the screen display portions215 a, 215 b, 450, 901 based on the deviation between the screen displayportions 215 a, 215 b, 450, 901.

In an embodiment, a method of operating an adjustment device 500including a case 510, 710, 810 in which a wearable electronic device200, 300, 530, 630, 730, 800, 900 including screen display portions 215a, 215 b, 450, 901 for a left eye and a right eye of a user and eyetracking cameras for the left eye and the right eye (e.g., the first eyetracking camera 270 a and the second eye tracking camera 270 b) isseated may include transmitting a virtual image 610 for measuring adeviation between the screen display portions 215 a, 215 b, 450, 901 tothe screen display portions 215 a, 215 b, 450, 901, projecting thevirtual image 610 onto a portion 511, 605, 711 of the case 510, 710, 810through a focal lens disposed in the case 510, 710, 810, capturing theprojected virtual images 610 by the first eye tracking camera 270 a andthe second eye tracking camera 270 b, and adjusting the screen displayportions 215 a, 215 b, 450, 901 based on a comparison result of thecaptured images.

The adjusting of the screen display portions 215 a, 215 b, 450, 901 mayinclude calculating the deviation between the screen display portions215 a, 215 b, 450, 901 by performing image processing on the capturedimages, and adjusting the screen display portions 215 a, 215 b, 450, 901based on the deviation.

The calculating of the deviation between the screen display portions 215a, 215 b, 450, 901 may include calculating a number of PIMA linesdisplayed in each of the captured images and a width of each of the PIMAlines, and calculating the deviation between the screen display portions215 a, 215 b, 450, 901 based on a result of comparing the number andwidth of the PIMA lines to a threshold.

In an embodiment, the method may further include determining whether toadjust the deviation between the screen display portions 215 a, 215 b,450, 901 based on the result of the comparing, and performingcalibration on the screen display portions 215 a, 215 b, 450, 901 basedon the determining.

In an embodiment, the adjusting of the screen display portions 215 a,215 b, 450, 901 may include at least one of adjusting the screen displayportions 215 a, 215 b, 450, 901 based on the deviation using a shaftprovided in a rim 910 or each of a left temple and a right temple of thewearable electronic device 200, 300, 530, 630, 730, 800, 900 and adriving device 813, 920, 930, 1010 installed in a stator 513, 713 of thecase 510, 710, 810, and adjusting the screen display portions 215 a, 215b, 450, 901 based on the deviation using a driving device 813, 920, 930,1010 provided in each of the screen display portions 215 a, 215 b, 450,901.

In an embodiment, each of the screen display portions 215 a, 215 b, 450,901 may be fixed to the rim 910 of the wearable electronic device 200,300, 530, 630, 730, 800, 900 by at least one of a magnet 920, a coil930, and a spring 940. The adjusting of the screen display portions 215a, 215 b, 450, 901 may include adjusting the screen display portions 215a, 215 b, 450, 901 based on the deviation using at least one of themagnet 920, the coil 930, and the spring 940 that fix each of the screendisplay portions 215 a, 215 b, 450, 901.

In an embodiment, the adjusting of the screen display portions 215 a,215 b, 450, 901 may include adjusting the screen display portions 215 a,215 b, 450, 901 based on the deviation by driving devices 813, 920, 930,1010 installed on a left rim and a right rim (e.g., rims 910) of thewearable electronic device 200, 300, 530, 630, 730, 800, 900.

In an embodiment, the wearable electronic device 200, 300, 530, 630,730, 800, 900 may further include displays 205, 210, 320, 410, 535, 735for the left eye and the right eye including micro-reflection mirrorsand projection lenses 325 and 415. The adjusting of the screen displayportions 215 a, 215 b, 450, 901 may include adjusting at least one of anoutput direction and a view angle of light of the projection lenses 325and 415 using the micro-reflection mirrors and adjusting focal distancesof the screen display portions 215 a, 215 b, 450, 901 based on thedeviation.

In an embodiment, the displays 205, 210, 320, 410, 535, 735 of thewearable electronic device 200, 300, 530, 630, 730, 800, 900 may includea display adjustable to a multiple focal plane (refer to FIG. 13 ). Theadjusting of the screen display portions 215 a, 215 b, 450, 901 mayinclude adjusting the multifocal plane through phase modulation of thedisplays 205, 210, 320, 410, 535, 735 and adjusting focal distances ofthe screen display portions 215 a, 215 b, 450, 901 based on thedeviation.

What is claimed is:
 1. An adjustment device for virtual imagescorresponding to a left eye and a right eye of a user, the adjustmentdevice comprising: a wearable electronic device which displays thevirtual images, the wearable electronic device comprising: displayapparatuses which correspond to the left eye and the right eye of theuser and display the virtual images; screen display portions whichcorrespond to the left eye and the right eye and transmit light sourcesgenerated by the display apparatuses to the left eye and the right eye;and eye tracking cameras which correspond to the left eye and the righteye; and a case in which the wearable electronic device is disposed, thecase comprising: a stator which fixes the wearable electronic device;and a focal lens which is disposed within an eye relief of the fixedwearable electronic device and forms each of images of the virtualimages output from the screen display portions of the wearableelectronic device on a portion of the case.
 2. The adjustment device ofclaim 1, wherein a portion of an interior of the case comprises a flatsurface without a curvature, and portions other than the portion of thecase are matt-coated or black-painted.
 3. The adjustment device of claim1, wherein the wearable electronic device captures images obtained byprojecting the virtual images and measures a deviation between thescreen display portions onto a portion of the case, using the eyetracking cameras, and adjusts the screen display portions based on acomparison result of the captured images.
 4. The adjustment device ofclaim 3, wherein the wearable electronic device further comprises: aprocessor which performs image processing on the captured images andcalculates the deviation between the screen display portions; and adriving device which adjusts the screen display portions based on thedeviation, and the processor calculates a number of lines of a virtualimage, measures a deviation and a width of each of the lines from eachof the captured images, determines whether to adjust the deviationbetween the screen display portions based on a result of comparing thenumber and the width of the lines of the virtual image to apredetermined threshold, and performs calibration on the left eye andthe right eye of the screen display portions based on the determining.5. The adjustment device of claim 1, wherein a shaft is provided in eachof left and right outer rims, or each of left and right temples of thewearable electronic device, and the stator further comprises a drivingdevice coupled to the shaft to adjust the screen display portions. 6.The adjustment device of claim 1, wherein the wearable electronic devicefurther comprises a driving device provided in each of the screendisplay portions, and the wearable electronic device adjusts the screendisplay portions based on a deviation between the screen displayportions, using the driving device.
 7. The adjustment device of claim 6,wherein the screen display portions are fixed to a left rim and a rightrim of the wearable electronic device, respectively, by at least one ofa magnet, a coil, and a spring, and each of the screen display portionsis adjusted based on the deviation by the at least one of the magnet,the coil, and the spring that fix each of the screen display portions.8. The adjustment device of claim 6, wherein driving devices areinstalled on a left rim and a right rim of the wearable electronicdevice, and the screen display portions are adjusted by the drivingdevices based on the deviation.
 9. The adjustment device of claim 1,wherein the display apparatuses further comprise micro-reflectionmirrors and projection lenses, the wearable electronic device adjusts atleast one of an output direction and a view angle of light of theprojection lenses using the micro-reflection mirrors, and adjusts focaldistances of the screen display portions based on a deviation betweenthe screen display portions.
 10. The adjustment device of claim 1,wherein the wearable electronic device further comprises: a sensor whichsenses whether a shaft of the wearable electronic device is fastened toa hole of the driving device.
 11. The adjustment device of claim 1,wherein a screen display portion of the screen display portions isadjustable by a first gear unit which is disposed on a side of atransparent member including the screen display portion and connectablewith a second gear unit of a driving device of the driving devices. 12.A method of operating an adjustment device comprising a case, in which awearable electronic device comprising screen display portionscorresponding to a left eye and a right eye of a user and eye trackingcameras corresponding to the left eye and the right eye are disposed,the method comprising: transmitting virtual images; measuring adeviation between the screen display portions to the screen displayportions; projecting the virtual images onto a portion of the casethrough focus lenses disposed in the case; capturing the projectedvirtual images by the eye tracking cameras; and adjusting the screendisplay portions based on a comparison result of the captured images.13. The method of claim 12, wherein the adjusting the screen displayportions comprises: calculating the deviation between the screen displayportions by performing image processing on the captured images; andadjusting the screen display portions based on the deviation.
 14. Themethod of claim 13, wherein the calculating the deviation comprises:calculating a number of Photographic and Imaging ManufacturersAssociation (“PIMA”) lines displayed in each of the captured images anda width of each of the PIMA lines; and calculating the deviation betweenthe screen display portions based on a result of comparing the numberand the width of the PIMA lines to a threshold.
 15. The method of claim14, further comprising: determining whether to adjust the deviationbetween the screen display portions based on the result of thecomparing; and performing calibration on the screen display portionsbased on the determining.
 16. The method of claim 12, wherein theadjusting the screen display portions comprises at least one of:adjusting the screen display portions based on the deviation using ashaft provided in a rim or each of a left temple and a right temple ofthe wearable electronic device and a driving device installed on astator of the case; and adjusting the screen display portions based onthe deviation, using a driving device provided in each of the screendisplay portions.
 17. The method of claim 16, wherein each of the screendisplay portions is fixed to the rim of the wearable electronic deviceby at least one of a magnet, a coil, and a spring, and the adjusting thescreen display portions comprises adjusting the screen display portionsbased on the deviation using the at least one of the magnet, the coil,and the spring that fix each of the screen display portions.
 18. Themethod of claim 16, wherein the adjusting the screen display portionscomprises adjusting the screen display portions based on the deviationby driving devices installed on a left rim and a right rim of thewearable electronic device.
 19. The method of claim 12, wherein thewearable electronic device further comprises: display apparatuses whichcorrespond to the left eye and the right eye and comprisemicro-reflection mirrors and projection lenses, and the adjusting thescreen display portions comprises: adjusting at least one of an outputdirection and a view angle of light of the projection lenses using themicro-reflection mirrors and adjusting focal distances of the screendisplay portions based on the deviation.
 20. A non-transitorycomputer-readable storage medium storing instructions that, whenexecuted by a processor, cause the processor to perform the method ofclaim 12.